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ABSTRACT 


Laboratory tests on complete systems have been a vital 
part of the effort to attain high reliability for earth satel- 
lites. The effectiveness of this approach is examined by 
summarizing the laboratory results on 64 spacecraft. These 
results are used to compare in-house and out-of-house per- 
formance with respect to the number of problems and en- 
vironments on both prototype and flight spacecraft. The in- 
house data are further developed to show the effect of time 
and thermal level on the detection of problems in a simu- 
lated space environment. Comparisons of space problems 
with simulated space problems are made with respect to 
number and location. The distribution of space problems 
with respect to time is presented and discussed. The space 
performance is used to show the merits of the test philoso- 
phy and test program, and also to suggest areas of 
improvement. 
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THE EFFECTIVENESS OF SYSTEMS TESTS IN ATTAINING 
RELIABLE EARTH SATELLITE PERFORMANCE* 

by 

A. R. Timmins 
Goddard Space Flight Centex 


INTRODUCTION 

Goddard Space Flight Center (GSFC) has been engaged in the design, development, test and evalu- 
ation, launching and tracking of unmanned earth satellites for approximately five years. During this 
time, many satellites have been tested and evaluated entirely at the Space Flight Center inGreenbelt. 

In addition, many unmanned sate Hites have been tested and evaluated in industry under the center’s 
guidance. Some data will be presented in summarized form which will utilize information from 
both sources. These data represent the broadest base from which to examine the number of 
problems, the environments in which the problems are detected, and the relationship between 
problems detected in the prototype and flight model spacecraft. (A problem is defined as any item 
that causes rework or delays during the qualification and acceptance testing of spacecraft.) The 
data from the two sources are compared in a gross way for any significant differences. 

All of the data are based on systems tests of complete spacecraft. One measure of the effective- 
ness of the system test is the ratio of problems detected in the prototype model spacecraft to those 
detected in the flight model. 

A general understanding of the test philosophy used at GSFC is needed before discussing the 
effectiveness of the systems tests. In brief, the engineering and prototype model spacecraft are 
used to prove that the design is satisfactory, with a safety margin on the order of 50 percent. Thus, 
the problems associated with design, quality control, materials, and operating procedures are 
evaluated in this phase. On the other hand, the flight model spacecraft are tested under simulated 
environments at the expected stress levels. These tests are used to detect workmanship problems 
and early failures, and to assure an acceptable level of confidence prior to the launch of the flight 
spacecraft. See details of the test philosophy, t 

Another manner of determining the effectiveness of systems tests is to compare the test re- 
sults of the flight model spacecraft with its actual space performance. Data from the first ten 
spacecraft tested and evaluated at GSFC are used for this comparison. The same data are then used 
to examine the relationship of space problems with their time of occurrence. 

^Presented at The Institute of Environmental Sciences, San Diego, California, April 1966 

t Boeckel, J.H., “The Purposes of Environmental Testing for Scientific Satellites/’ NASA Technical Note D-1900, 1963. 
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Finally, the problems encountered in simulated space tests are examined with respect to the 
time and thermal level at which the problems were detected. From these data, a judgment is made 
as to the time required for testing a spacecraft to attain reliable space performance. 

LABORATORY TEST RESULTS 

Figure 1 presents summarized data obtained from the testing of 64 spacecraft— 16 of these 
being prototype models and 48 being flight models. It shows a ratio of approximately 4-to-l for 
the number of problems per spacecraft on the prototype models versus the flight models. This 
ratio indicates the importance of using a prototype spacecraft and is a measure of the effectiveness 
of the systems test. For instance, if most problems are detected and corrected in the prototype 
systems test, there should be few in the flight model spacecraft. Under these conditions, a large 
ratio is desirable. While there are no fixed values for the ratio or its parts, Figure 1 does give 
some values which can be used for comparison with similar data. The large number of problems 
per prototype spacecraft (about 30) is also indicative of the need for this type of spacecraft. 

Figure 2 presents the same kind of information as shown in Figure 1, except that it is sub- 
divided to show the performance at GSFC compared to the performance in industry. The ratio of 
prototype to flight model problems is 5-to-l for the in-house programs and 3-to-l for the out-of- 
house programs. No attempt is made at present to attach significance to this difference in ratios. 
The value of the ratios will be to give management a means of comparing past, present, and future 
programs. 

Figure 3 shows the distribution of the problems by the simulated environment in which the 
problem occurred. The simulated space environment is the major contributor to problems during 
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Figure 1— Spacecraft test problems 
(prototype vs. flight model). 


Figure 2— Spacecraft test problems 
(in-house vs. out- of- house). 





the prototype systems tests. A desirable 
learning curve is demonstrated for this en- 
vironment by comparing the results on pro- 
totype and flight model systems. The vibration 
systems tests show about two problems per 
spacecraft on flight models and six to eight on 
prototype spacecraft. This desirable result on 
vibration systems tests is partly attributable 
to the earlier elimination of some problems 
through testing of a structural model in some 
programs. 

In Figure 4 the distribution of the problems 
per spacecraft among the most prominent sub- 
systems of the satellite is shown, and also the 
ratio of prototype to flight models problems 
for each major subsystem. In addition, the 
simulated environment in which the problems 
were detected is shown. 
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Figure 3— Spacecraft test problems (by environment). 
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Figure 4— In-house spacecraft test problems (as related to environment). 


SPACE TEST RESULTS 


There is no universal method for measuring satellite performance. For zero or 100 percent 
performance, there is no problem; but between these limits, values become qualitative and it is 
difficult to compare satellites. For instance, timeliness and amount of data are not comparable; 
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likewise the value of one kind of data versus another depends on the interest of the experimenter. 
For our purpose, a severe qualitative system will be used to examine space results. Any perform- 
ance outside the planned limits of the spacecraft or program is defined as a problem. In other 
words, all failures are included; but other items are also included, even though they did not result 
in loss of data. For instance, if the sun aspect angle of the spacecraft was beyond the prescribed 
limits, it was considered a problem, even though no malfunction or loss of data resulted. This 
approach admittedly tends to underrate the space performance, which in the cases to be discussed 
was excellent. 

Figure 5 compares space problems with test problems for ten spacecraft tested and evaluated 
at GSFC. A wide range (1 to 28) of problems has been detected on the individual flight model space- 
craft in laboratory tests, while the range of problems in space on the same spacecraft has been 
gratifyingly small (one to six). At the same time, it is apparent that we have not achieved per- 
fection. These results raise a natural question: What parts of a satellite cause the most problems? 

Figure 6 gives an answer to this question. It shows the major subsystems of a spacecraft 
with the number of problems detected in the laboratory compared with problems encountered in 
space. The experiments are shown to be the major contributor to the problems— both in the 
laboratory and in space. This is not too surprising, since most experiments use advanced state- 
of-the-art hardware, and are much smaller in weight and size than similar items attempting to make the 
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Figure 5— Comparison of space problems with test problems for ten GSFC spacecraft. 
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AVERAGE PROBLEMS PER FLIGHT SPACECRAFT 
Figure 6— Summary of space problems vs. laboratory problems. 
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same measurements on earth. It should also be noted that multiple experiments are flown on most 
spacecraft. Fortunately, the failure of one experiment does not usually affect the operation of the 
spacecraft or other experiments. The "other” category in Figure 6 is large because it includes 
many kinds of problems. For example, the laboratory "other" category includes thermal, con- 
nector, corona, facility, and radio frequency interference problems. 

Figure 7 shows the distribution of problems in space versus time for the same ten GSFC 
spacecraft which have been under discussion. Here we do find a surprising number of first-day 
problems. The distribution of problems with time after the first day shows no pronounced pattern. 
Also, the number of spacecraft shown in operation at the end of a year, although small, indicates 
that one-year operation of unmanned earth satellites is quite possible. (In fact, Alouette I has 
demonstrated that three -year life is attainable.) 

The first day-problems shown in Figure 7 might seem to indicate a launch environment more 
severe than was expected, or possibly a combined environment which could not be simulated in the 
test program. Each of the problems has been investigated extensively, and the following general 
remarks are pertinent: Five of the problems have been sufficiently identified as not caused by the 
launch environment. Four could not be ascribed to a specific cause; of these, three were of a 
nature which could have been attributed to the launch environment. Thus, the launch environment 
does not account for all of the first-day failures, although it must still be considered a critical 
stage of the satellite’s life. Further, additional emphasis will be given to this environment at GSFC 
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DAYS 

INCLUSION OF THREE PROBE LAUNCHES ACCOUNTS FOR 
INITIAL DROP TO SEVEN SPACECRAFT 

Figure 7- Problems in space vs. time (ten GSFC spacecraft). 

A Launch Phase Simulator now under construction will permit full-scale combined environment 
testing. The combined environments of launch acceleration, three degree-of -freedom vibration 
testing, acoustic loading, and vacuum will be available in this facility. 

DISCUSSION 

The large number of problems shown in Figure 7 raises the question whether the flight model 
spacecraft were tested for a sufficient length of time. Figure 8 presents summarized data on this 
subject. It shows the cumulative number of problems versus time for seven flight model space- 
craft in a simulated space environment. With this type of presentation, the time to reach a plateau 
should be considered the minimum test time. If the temperature levels used for testing had no effect 
on the number of malfunctions, then the curve labeled "total” would be indicative of the total test 
time required. (The total curve was constructed by plotting the sum of the first-day hot and first- 
day cold problems at two days, etc.). However, the curve which depicts the effect of time and tem- 
perature shows a pronounced temperature effect. This curve was constructed by plotting all of the 
first day hot failures at day one even though the first exposure to the hot thermal level may have 
been at day 3, 4, or later. The sequence of the hot curve before the cold curve is arbitrary. The 
plateau appears to be reached in about six days for each thermal level. This should be considered 
a minimum time because not all of the spacecraft were tested for the full time. When correlation 
of the laboratory test time with the first-day space failures is attempted, there is no consistency. 

In other words, two of the spacecraft with the longest test times had first-day problems in space, 
whereas two with shorter test times did not have early space failures. The lack of consistency 
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Figure 8— Problems in simulated space environment vs. time. 


leads to questions as to the causes of the problems in space and in the laboratory, and also to 
what other test techniques may be helpful. 

For the period of time covered by this report, there has been insufficient failure analysis or 
documentation to adequately ascribe causes of failures. Thus, the data in Figure 8 probably con- 
sist of both environmentally induced failures and failures unrelated to the environment. When the 
latter can be screened out, the test times will be more realistic; and the additional information 
should be useful in showing what kinds of control need to be emphasized. Another inference 
from Figure 8 should be noted. The pronounced effect of the temperature -time environment 
suggests that other kinds of temperature-time relationships may be important in eliminating more 
problems in the test phase. Such temperature -time relationships as repeated temperature cycling 
and maximum temperature gradients have not been fully explored in all of the spacecraft included 
in the data. The number of such problems is expected to be small. For instance, in one case, a solar 
simulation test was conducted after a conventional thermal-vacuum test of twelve days. A thermal- 
coating adhesion problem was found, although no additional spacecraft problems were detected. 

In conclusion, our experience indicates that a minimum of six days at each thermal extreme 
is required to assure that major system defects are revealed. In addition, the test(s) should in- 
clude simulation of high rate of change of temperatures as well as simulation of large thermal 
gradients when applicable to specific spacecraft. 

A review of the problems depicted in Figure 7 revealed additional areas in which the informa- 
tion gained will be helpful in further improving space performance: 

(a) In about 25 percent of the space problems, the spacecraft were not subjected to relevant 
systems tests. For example, in one case the nose cone outgassed and changed the properties of 
the satellite's thermal coating. This caused overheating and failure of an experiment. 
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(b) In some cases a relevant systems test was not possible. For instance, no simulation 
capability is available for testing stability problems caused by solar radiation pressure, or aero- 
dynamic forces at perigee altitudes. 

(c) Flight devices which cannot be operated during test require special attention. Explosively 
actuated devices are an example. On one launch an antenna failed to deploy; post-launch investi- 
gation revealed a marginal firing current for the explosively actuated device used for deployment. 

The review of the actual and simulated space performance has indicated areas in which 
the number of problems in space can be reduced. However, the space performance review also 
shows one kind of problem that does not have a ready explanation or answer— that is, the problem 
that is apparently time-dependent and occurs only after a long period in space. 

Before the effectiveness of the systems test is assessed, a closer look at the space perform- 
ance is needed. Our critical approach and definition of a problem shows some 30 problems in space 
on the ten spacecraft under review. This would seem to indicate questionable or poor performance, 
but such is not the case. Each spacecraft has been successful in providing scientific data on the 
space environment. In some cases less than 100 percent of the desired data have been obtained. 
However, in these cases only one (or more) of several experiments malfunctioned, or the space- 
craft did not attain its planned life. Seven of the ten spacecraft attained or exceeded the planned 
lifetime. The other three had lifetimes of 112, 193, and 312 days. One of these (193 days) pro- 
vided nearly 5000 hours of scientific information which has led to 16 scientific papers to date. 

Thus one can perceive that the value or success of a satellite is not necessarily measured by the 
hours of operation or kilobits of data, but rather the knowledge that has been gained. From this 
standpoint the space performance of the ten satellites has been eminently successful. 

Just as there is no satisfactory number system with which to measure the space performance 
of a satellite, there is no quantitative measure of the effectiveness of the systems tests. However, 
some insight on what these tests have contributed and some of their limitations can be gained from 
these data. By utilizing the GSFC data, the effectiveness and limitations of the systems tests may 
be summarized as follows: 

1. The significant decrease in the number of problems in the flight spacecraft compared to 
the prototype spacecraft (Figures 1 and 2). 

2. The detection of 216 problems on seven prototype spacecraft in the systems tests. Of these, 
seven were classified as catastrophic, 75 as major, and 134 as minor problems. 

3. The detection of 97 problems on ten flight spacecraft in the systems tests. Of these, two 
were classified as catastrophic, 43 were major, and 52 were minor problems. 

4. The operation of each of the ten satellites in space. Seven of the ten provided some scientific 
data for the entire planned lifetime. All returned data which have increased our understanding of 
the space environment. 

5. Workmanship. Some 20 percent of the problems encountered in the systems tests were 
attributed to faulty workmanship. In the future, reduction of this type of problem can be expected 



through greater emphasis on procurement specifications, high reliability parts, inspection, and 
other quality assurance provisions. 

6. Some space problems, apparently time-dependent, are not detected in short-term simulated 
space tests. 


Goddard Space Flight Center 
National Aeronautics and Space Administration 
Greenbelt, Maryland, June 9, 1966 
124-12-03-01-51 
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"The aeronautical and space activities of the United States shall be 
conducted so as to contribute ... to the expansion of human knowl- 
edge of pheizomena in the atmosphere and space v The Administration 
shall provide for the widest practicable and appropriate dissemination 
of information concerning its activities and the results t hereof.” 

— National Aeronautics and Space Act of 1958 
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